Computational Imaging: a Quest for the Perfect Image

A physical lens is limited in its ability to capture an image that is both high- resolution and wide-field due to aberrations even with a sophisticated lens design. This thesis explores computational methods that expand on the recently developed Fourier ptychographic microscopy (FPM) to overcome the physical limitations. New algorithms and imaging methods extend the computational aberration correction to more general imaging modalities including fluorescence microscopy and incoherent bright-field imaging so as to allow even a crude lens to perform like an ideal lens. This paradigm shift from the lens design to computational algorithms democratizes high-resolution imaging by making it easier to use and less complicated to build

Fourier ptychographic reconstruction using Poisson maximum likelihood and truncated Wirtinger gradient

Fourier ptychographic microscopy (FPM) is a novel computational coherent imaging technique for high space-bandwidth product imaging. Mathematically, Fourier ptychographic (FP) reconstruction can be implemented as a phase retrieval optimization process, in which we only obtain low resolution intensity images corresponding to the sub-bands of the sample's high resolution (HR) spatial spectrum, and aim to retrieve the complex HR spectrum. In real setups, the measurements always suffer from various degenerations such as Gaussian noise, Poisson noise, speckle noise and pupil location error, which would largely degrade the reconstruction. To efficiently address these degenerations, we propose a novel FP reconstruction method under a gradient descent optimization framework in this paper. The technique utilizes Poisson maximum likelihood for better signal modeling, and truncated Wirtinger gradient for error removal. Results on both simulated data and real data captured using our laser FPM setup show that the proposed method outperforms other state-of-the-art algorithms. Also, we have released our source code for non-commercial use

Aperture scanning Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is implemented through aperture scanning by an LCOS spatial light modulator at the back focal plane of the objective lens. This FPM configuration enables the capturing of the complex scattered field for a 3D sample both in the transmissive mode and the reflective mode. We further show that by combining with the compressive sensing theory, the reconstructed 2D complex scattered field can be used to recover the 3D sample scattering density. This implementation expands the scope of application for FPM and can be beneficial for areas such as tissue imaging and wafer inspection

A STUDY ON THE OPTIMAL DEFENSE POSITION OF HANDBALL GOALKEEPER: FACING A FORWARD JUMP SHOT MADE FROM 9M

The purpose of the study was to derive an optimal defense position for goalkeepers through calculating ideal defense area and actual defense area when blocking forward jump shots from 9 m distance. A total of 9 men's handball matches were captured with 4 digital video cameras from the 94th Korea National Sports Festival with consent from the Korea Handball Federation. Video clips with 78 forward jump shots from 9 m distance which included the whole movements of goalkeeper were analyzed. The ball's speed in the forward jump shot, release point of the ball, goalkeeper's defensive stance, and defensive position were analyzed. The actual defense position of goalkeeper was 1.10*0.37 m and the optimal defense position calculated through the movements of goalkeeper was 1.45 m from the goal line

Counting White Blood Cells from a Blood Smear Using Fourier Ptychographic Microscopy

White blood cell (WBC) count is a valuable metric for assisting with diagnosis or prognosis of various diseases such as coronary heart disease, type 2 diabetes, or infection. Counting WBCs can be done either manually or automatically. Automatic methods are capable of counting a large number of cells to give a statistically more accurate reading of the WBC count of a sample, but the specialized equipment tends to be expensive. Manual methods are inexpensive since they only involve a conventional light microscope setup. However, it is more laborious and error-prone because the small field-of-view (FOV) of the microscope necessitates mechanical scanning of a specimen for counting an adequate number of WBCs. Here, we investigate the use of Fourier ptychographic microscopy (FPM) to bypass these issues of the manual methods. With a 2x objective, FPM can provide a FOV of 120 mm^2 with enhanced resolution comparable to that of a 20x objective, which is adequate for non-differentially counting WBCs in just one FOV. A specialist was able to count the WBCs in FPM images with 100% accuracy compared to the count as determined from conventional microscope images. An automatic counting algorithm was also developed to identify WBCs from FPM’s captured images with 95% accuracy, paving the way for a cost-effective WBC counting setup with the advantages of both the automatic and manual counting methods

Wide-field Fourier ptychographic microscopy using laser illumination source

Fourier ptychographic (FP) microscopy is a coherent imaging method that can synthesize an image with a higher bandwidth using multiple low-bandwidth images captured at different spatial frequency regions. The method’s demand for multiple images drives the need for a brighter illumination scheme and a high-frame-rate camera for a faster acquisition. We report the use of a guided laser beam as an illumination source for an FP microscope. It uses a mirror array and a 2-dimensional scanning Galvo mirror system to provide a sample with plane-wave illuminations at diverse incidence angles. The use of a laser presents speckles in the image capturing process due to reflections between glass surfaces in the system. They appear as slowly varying background fluctuations in the final reconstructed image. We are able to mitigate these artifacts by including a phase image obtained by differential phase contrast (DPC) deconvolution in the FP algorithm. We use a 1-Watt laser configured to provide a collimated beam with 150 mW of power and beam diameter of 1 cm to allow for the total capturing time of 0.96 seconds for 96 raw FPM input images in our system, with the camera sensor’s frame rate being the bottleneck for speed. We demonstrate a factor of 4 resolution improvement using a 0.1 NA objective lens over the full camera field-of-view of 2.7 mm by 1.5 mm

Simultaneous fluorescence and high-resolution bright-field imaging with aberration correction over a wide field-of-view with Fourier ptychographic microscopy (FPM)

We present a method to acquire both fluorescence and high-resolution bright-field images with correction for the spatially varying aberrations over a microscope’s wide field-of-view (FOV). First, the procedure applies Fourier ptychographic microscopy (FPM) to retrieve the amplitude and phase of a sample, at a resolution that significantly exceeds the cutoff frequency of the microscope objective lens. At the same time, FPM algorithm is able to leverage on the redundancy within the set of acquired FPM bright-field images to estimate the microscope aberrations, which usually deteriorate in regions further away from the FOV’s center. Second, the procedure acquires a raw wide-FOV fluorescence image within the same setup. Lack of moving parts allows us to use the FPM-estimated aberration map to computationally correct for the aberrations in the fluorescence image through deconvolution. Overlaying the aberration-corrected fluorescence image on top of the high-resolution bright-field image can be done with accurate spatial correspondence. This can provide means to identifying fluorescent regions of interest within the context of the sample’s bright-field information. An experimental demonstration successfully improves the bright-field resolution of fixed, stained and fluorescently tagged HeLa cells by a factor of 4.9, and reduces the error caused by aberrations in a fluorescence image by 31%, over a field of view of 6.2 mm by 9.3 mm. For optimal deconvolution, we show the fluorescence image needs to have a signal-to-noise ratio of ~18

Diffraction tomography with Fourier ptychography

This paper presents a technique to image the complex index of refraction of a sample across three dimensions. The only required hardware is a standard microscope and an array of LEDs. The method, termed Fourier ptychographic tomography (FPT), first captures a sequence of intensity-only images of a sample under angularly varying illumination. Then, using principles from ptychography and diffraction tomography, it computationally solves for the sample structure in three dimensions. The experimental microscope demonstrates a lateral spatial resolution of 0.39 μm and an axial resolution of 3.7 μm at the Nyquist–Shannon sampling limit (0.54 and 5.0 μm at the Sparrow limit, respectively) across a total imaging depth of 110 μm. Unlike competing methods, this technique quantitatively measures the volumetric refractive index of primarily transparent and contiguous sample features without the need for interferometry or any moving parts. Wide field-of-view reconstructions of thick biological specimens suggest potential applications in pathology and developmental biology

Overlapped Fourier coding for optical aberration removal

We present an imaging procedure that simultaneously optimizes a camera’s resolution and retrieves a sample’s phase over a sequence of snapshots. The technique, termed overlapped Fourier coding (OFC), first digitally pans a small aperture across a camera’s pupil plane with a spatial light modulator. At each aperture location, a unique image is acquired. The OFC algorithm then fuses these low-resolution images into a full-resolution estimate of the complex optical field incident upon the detector. Simultaneously, the algorithm utilizes redundancies within the acquired dataset to computationally estimate and remove unknown optical aberrations and system misalignments via simulated annealing. The result is an imaging system that can computationally overcome its optical imperfections to offer enhanced resolution, at the expense of taking multiple snapshots over time